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Congressional  Committees 

Defense  Nuclear  Enterprise:  DOD  Has  Established  Processes  for  Implementing  and 
Tracking  Recommendations  to  Improve  Leadership,  Morale,  and  Operations 

The  Department  of  Defense  (DOD)  has  identified  nuclear  deterrence  as  its  highest  priority 
mission.  In  2014,  in  response  to  several  incidents  involving  the  nation’s  nuclear  deterrent  forces 
and  their  senior  leadership,  the  Secretary  of  Defense  directed  two  reviews  of  DOD’s  nuclear 
enterprise.  These  two  reviews  identified  problems  with  leadership,  organization,  investment, 
morale,  policy,  and  procedures,  as  well  as  other  shortcomings  that  are  adversely  affecting  the 
nuclear  deterrence  mission.  The  reviews  also  made  recommendations  to  address  these 
problems. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2016  includes  a  provision  for  us  to 
review  DOD’s  process  for  addressing  the  recommendations  of  the  nuclear  enterprise  reviews.1 
In  addition,  Senate  Report  1 14-49  includes  a  provision  for  us  to  review  certain  matters  related  to 
DOD’s  approach  for  addressing  the  recommendations.2  This  report  describes  the  extent  to 
which  DOD  has  established  processes  to  (1)  implement  the  recommendations  of  the  two 
nuclear  enterprise  reviews  and  (2)  track  the  implementation  of  these  recommendations  and 
measure  the  effectiveness  of  the  actions  it  has  taken  to  address  them.  We  briefed  the 
congressional  defense  committees  in  February  2016.  This  report  formally  transmits  updated 
briefing  slides  regarding  the  results  of  our  work  (see  enclosure). 

To  identify  DOD’s  processes  for  implementing  the  recommendations,  tracking  the  progress  of 
the  implementation,  and  measuring  the  effectiveness  of  actions  taken,  we  reviewed  key 
documents,  including  the  reports  of  the  nuclear  enterprise  reviews,  Strategic  Command’s  action 
plan,  DOD  guidance  (including  the  Secretary  of  Defense  memo  Nuclear  Enterprise  Review 
Corrective  Action  Implementation),  and  documents  from  the  Office  of  Cost  Assessment  and 
Program  Evaluation  (CAPE)  and  the  services  that  track  the  status  of  the  recommendations.  We 
also  interviewed  DOD  officials  regarding  the  processes  the  department  and  the  services  have 
developed.  We  compared  the  processes  to  relevant  criteria  from  Standards  for  Internal  Control 
in  the  Federal  Government — including  assessing  and  responding  to  risk,  using  and  effectively 
communicating  quality  information,  and  performing  monitoring  activities. 3  The  focus  of  this 


^ee  Pub.  L.  No.  114-92,  §  1658  (2015). 

2 See  S.  Rep.  No.  114-49,  at  269  (2015). 

3GAO,  Standards  for  Internal  Control  in  the  Federal  Government ,  GAO-14-704G  (Washington,  D.C.:  September 
2014).  These  standards  went  into  effect  in  October  2015,  and  we  used  them  to  assess  DOD’s  activities  since  that 
time. 
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review  was  on  actions  taken  in  response  to  the  recommendations  of  the  two  2014  nuclear 
enterprise  reviews;  we  did  not  review  the  methodology  used  to  conduct  those  initial  reviews. 

We  conducted  this  performance  audit  from  July  2015  to  July  2016  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained  provides 
a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives. 

In  summary,  we  found  that  DOD  has  established  a  process  for  implementing  the  two  nuclear 
enterprise  reviews’  recommendations.  In  November  2014,  the  Secretary  of  Defense  directed 
CAPE  to  track  and  assess  the  implementation  of  the  recommendations  from  the  two  nuclear 
enterprise  reviews.  The  Joint  Staff,  Navy,  Air  Force,  offices  within  the  Office  of  the  Secretary  of 
Defense,  and  Strategic  Command  are  supporting  CAPE’s  efforts.  CAPE  compiled  the 
recommendations  from  the  two  reviews  and  a  memorandum  from  the  Commander  of  U.S. 
Strategic  Command  that  identified  several  additional  recommendations.  CAPE  identified  175 
distinct  recommendations  from  the  three  documents.  CAPE  then  identified  247  sub¬ 
recommendations  from  recommendations  directed  to  multiple  services  (or  other  DOD 
components).  CAPE  then  worked  with  the  services  to  identify  offices  of  primary  responsibility 
for  implementing  actions  to  address  the  recommendations,  any  offices  of  coordinating 
responsibility,  and  any  resources  necessary  to  implement  each  recommendation. 

CAPE  has  developed  a  tracking  tool  to  collect  information  on  progress  in  meeting  milestones 
and  metrics.  This  tracking  tool  identifies  offices  of  responsibility,  implementation  actions, 
milestones,  and  metrics  to  measure  the  effectiveness  of  the  actions  taken  for  each  of  the 
recommendations.  The  tracking  tool  currently  contains  hundreds  of  unique  milestones  and 
metrics,  and  additional  milestones  and  metrics  are  still  being  identified.  The  Air  Force  and  Navy 
also  have  their  own  methods  of  tracking  their  service-specific  recommendations.  The  services 
provide  CAPE  with  information  to  update  CAPE’s  centralized  tracking  tool.  As  needed,  CAPE 
requests  clarification  of  information  provided  by  the  services  and  other  DOD  components.  CAPE 
also  provides  the  components  with  updated  information  on  the  status  of  the  milestone  and 
metrics  from  the  centralized  tracking  tool  and  data  independently  gathered  to  support  the 
centralized  tracking  tool.  As  of  April  2016,  CAPE  reported  that  20  of  the  sub-recommendations 
were  closed,  28  had  been  identified  as  complete  by  the  responsible  DOD  component  and  were 
in  the  process  of  being  reviewed  for  possible  closure,  and  199  were  in  progress. 

Information  from  CAPE’s  centralized  tracking  tool  is  used  to  inform  a  group  of  senior  officials, 
known  as  the  Nuclear  Deterrence  Enterprise  Review  Group  (NDERG),  which  oversees  the 
department’s  progress.  The  processes  DOD  has  developed  generally  appear  consistent  with 
relevant  criteria  within  the  Standards  for  Internal  Control  in  the  Federal  Government — including 
using  and  effectively  communicating  quality  information  and  performing  monitoring  activities.4 
CAPE  has  also  begun  to  include  information  related  to  assessing  and  responding  to  risk  within 
its  tracking  process. 

We  are  not  making  any  recommendations  in  this  report. 


4GAO-14-704G. 
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Agency  Comments 

We  provided  a  draft  of  this  report  to  DOD  for  review  and  comment.  DOD  provided  technical 
comments  which  we  incorporated  as  appropriate. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional  committees;  the  Secretary 
of  Defense;  the  Secretaries  of  the  Navy  and  Air  Force;  the  Commander,  Strategic  Command; 
and  the  Director  of  CAPE.  In  addition,  the  report  is  available  at  no  charge  on  the  GAO  website 
at  http://www.gao.gov. 


If  you  or  your  staff  have  questions  concerning  this  report,  please  contact  me  at  (202)  512-9971 
or  kirschbaumj@gao.gov.  Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
Affairs  may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to 
this  report  include  Penney  Harwell  Caramia,  Assistant  Director;  Carly  Gerbig;  Brent  Helt; 
Joanne  Landesman;  Amie  Lesser;  and  Mike  Shaughnessy. 


Joseph  W.  Kirschbaum 

Director,  Defense  Capabilities  and  Management 
Enclosure 
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Enclosure:  Updated  Briefing  for  Congressional  Defense  Committees 


GAD 

Department  of  Defense  Nuclear 
Enterprise  Reviews 


Information  Provided  to  Congressional  Defense 

Committees 
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GAO 


Contents 


•  Background 

•  Objectives 

•  Scope  and  Methodology 

•  Summary 

•  The  Department  of  Defense  (DOD)  Has  Established  a  Process  for 
Implementing  Recommendations  from  the  Nuclear  Enterprise  Reviews 

•  The  Office  of  Cost  Assessment  and  Program  Evaluation  (CAPE)  and  the 
Services  Have  Established  Processes  to  Track  the  Implementation  of 
Recommendations,  and  They  Continue  to  Identify  Measures  of  Effectiveness 
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GAO 


Background 


•  The  Department  of  Defense  (DOD)  has  identified  nuclear  deterrence  as  its 
highest  priority  mission.  DOD’s  nuclear  enterprise  consists  of  Air  Force 
Intercontinental  Ballistic  Missiles;  nuclear-capable  bombers  and  tactical 
fighters;  Navy  ballistic  missile  submarines;  and  the  supporting  infrastructure  to 
build,  maintain,  and  control  these  assets. 

•  In  2014,  in  response  to  several  incidents  involving  the  nation’s  nuclear 
deterrent  forces  and  their  senior  leadership,  the  Secretary  of  Defense  directed 
two  reviews  of  DOD’s  nuclear  enterprise: 

•  An  independent  review  led  by  retired  General  Welch  and  retired  Admiral 
Harvey,  and 

•  An  internal  review  led  by  Assistant  Secretary  of  Defense  Creedon  and  Rear 
Admiral  Fanta. 

•  The  two  reviews  identified  problems  with  leadership,  organization,  investment, 
morale,  policy,  and  procedure,  as  well  as  other  shortcomings  that  are  adversely 
affecting  the  mission.  The  reviews  also  made  recommendations  to  address 
these  problems. 
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GAO 


Objectives 


The  National  Defense  Authorization  Act  for  Fiscal  Year  2016  includes  a  provision 
for  GAO  to  review  DOD’s  process  for  addressing  the  recommendations  of  the 
nuclear  enterprise  reviews.1  In  addition,  Senate  Report  114-49  includes  a  provision 
that  GAO  review  certain  matters  related  to  DOD’s  approach  for  addressing  the 
recommendations.2 

This  briefing  presents  our  preliminary  observations  on  the  extent  to  which  DOD 
has  established  processes  to 

1 .  Implement  the  recommendations  of  the  two  nuclear  enterprise  reviews  and 

2.  T rack  the  implementation  of  these  recommendations  and  measure  the 
effectiveness  of  the  actions  it  has  taken  to  address  them. 


^ee  Pub.  L.  No.  114-92,  §  1658  (2015). 

2See  S.  Rep.  No.  114-49,  at  269  (2015). 
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GAO 


Scope  and  Methodology 

To  evaluate  the  extent  to  which  DOD  has  established  a  process  to  implement  the 

recommendations  of  the  two  nuclear  enterprise  reviews,  we 

•  Reviewed  key  documents,  including  the  independent  and  internal  nuclear  enterprise  reviews,  Strategic 
Command’s  action  plan,  guidance  such  as  the  Secretary  of  Defense  memo  (Nuclear  Enterprise 
Review  Corrective  Action  Implementation ),  and  documents  from  DOD’s  Office  of  Cost  Assessment 
and  Program  Evaluation  (CAPE)  and  the  services  that  track  the  status  of  recommendations. 

•  Compared  DOD’s  implementation  process  to  relevant  criteria  from  Standards  for  Internal  Control  in  the 
Federal  Government—  including  assessing  and  responding  to  risk,  using  and  effectively 
communicating  quality  information,  and  performing  monitoring  activities.3 

To  evaluate  the  extent  to  which  DOD  has  established  a  process  to  track  implementation  and  measure  the 

effectiveness  of  actions  it  has  taken,  we 

•  Reviewed  CAPE’s  tracking  tool  for  the  Nuclear  Enterprise  Review  Tracking  (NERT)  team,  which  is 
used  to  record  progress  on  implementing  and  closing  recommendations.  It  is  also  used  to  record  the 
planned  approach  developed  by  the  nuclear  components  and  CAPE  for  assessing  the  effectiveness  of 
DOD’s  actions. 

•  Reviewed  the  Air  Force’s  and  Navy’s  separate  tracking  processes  and  compared  the  type  of 
information  they  track  to  that  contained  in  the  central  tracking  tool. 

3GAO,  Standards  for  Internal  Control  in  the  Federal  Government ,  GAO-14-704G  (Washington,  D.C.: 

September  2014).  These  standards  went  into  effect  in  October  2015,  and  we  used  them  to  assess  DOD’s 

activities  since  that  time. 
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GAO 

Scope  and  Methodology  (continued) 

We  interviewed  officials  from  the  following  offices: 

•  Under  Secretary  of  Defense  for  Policy 

•  Assistant  Secretary  of  Defense  for  Nuclear,  Chemical,  and  Biological  Defense 
Programs 

•  CAPE 

•  Joint  Staff  (J33,  J5) 

•  Air  Force  Headquarters  (A10) 

•  Air  Force  Global  Strike  Command 

•  Air  Force  Materiel  Command 

•  20th  Air  Force 

•  Air  Education  and  Training  Command 

•  Air  Force  Nuclear  Weapons  Center 

•  Air  Force  Inspector  General 

•  Deputy  Under  Secretary  of  the  Navy  for  Policy 

•  Chief  of  Naval  Operations  (N514,  N80,  N97) 

•  Strategic  Systems  Programs 

•  Naval  Reactors 

•  United  States  Strategic  Command 
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GAO 


Summary 


DOD  has  established  a  process  for  implementing  the  internal  and  external  nuclear 
enterprise  reviews’  recommendations.  This  process  generally  appears  consistent 
with  relevant  criteria  within  the  Standards  for  Internal  Control  in  the  Federal 
Government.  CAPE  worked  with  the  services  to  identify  offices  of  primary 
responsibility  for  implementing  actions  to  address  the  recommendations,  any 
offices  with  coordinating  responsibility,  and  any  resources  necessary  to  implement 
each  recommendation.  A  group  of  senior  officials,  known  as  the  Nuclear 
Deterrence  Enterprise  Review  Group  (NDERG),  which  includes  the  Deputy 
Secretary  of  Defense  and  Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  oversees  the 
department’s  progress  using  information  provided  by  CAPE  and  the  services. 

CAPE  and  the  services  have  established  processes  to  track  the  implementation  of 
the  nuclear  enterprise  reviews’  recommendations.  CAPE  has  developed  a  tracking 
tool  to  collect  information  on  progress  in  meeting  milestones  and  metrics.  The  Air 
Force  and  Navy  also  have  their  own  methods  of  tracking  their  service-specific 
recommendations.  The  services  provide  CAPE  with  information  to  update  CAPE’s 
centralized  tracking  tool. 
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GAD 

DOD  Has  Established  a  Process  for  Implementing  Recommendations  from 
the  Nuclear  Enterprise  Reviews 


•  In  November  2014,  the  Secretary  of  Defense  directed  DOD  to  address  the  recommendations 
from  the  two  nuclear  enterprise  reviews,  and  the  Secretary  directed  CAPE  to  track  and  assess 
the  implementation  of  the  recommendations.  The  Joint  Staff,  Navy,  Air  Force,  offices  within  the 
Office  of  the  Secretary  of  Defense,  and  Strategic  Command  support  CAPE’S  efforts. 

•  The  Secretary  also  established  the  NDERG,  a  group  of  senior  officials  chaired  by  the  Deputy 
Secretary  of  Defense  and  including  the  Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  to  oversee  and 
make  decisions  regarding  implementation  of  the  nuclear  enterprise  reviews’  recommendations. 

•  CAPE  compiled  recommendations  from  the  two  reviews  and  a  memorandum  from  the 
Commander  of  Strategic  Command,  which  identified  several  additional  recommendations.  CAPE 
identified  175  distinct  recommendations  from  the  three  documents.  CAPE  then  identified  247  sub¬ 
recommendations  from  recommendations  directed  to  multiple  services  (or  other  DOD 
components).  Table  1  provides  the  number  of  recommendations  by  category. 

•  CAPE  then  worked  with  the  services  to  identify  offices  of  primary  responsibility  for  implementing 
actions  to  address  the  recommendations,  any  offices  of  coordinating  responsibility,  and  any 
resources  necessary  to  implement  each  recommendation. 

•  These  activities  are  generally  consistent  with  the  principles  for  performing  monitoring  activities 
found  in  Standards  for  Internal  Control  in  the  Federal  Government— including  the  establishment 
of  a  baseline,  internal  control  system  monitoring,  and  evaluation  of  results. 
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GAO 

DOD  Has  Established  a  Process  for  Implementing  Recommendations  from 
the  Nuclear  Enterprise  Reviews  (continued) 


Table  1 :  Number  of  Recommendations  and  Sub-recommendations  by  Category 


Recommendation  Category 

Recommendations 

Sub-recommendations 

Operations  and  Maintenance 

30 

36 

Investment 

13 

17 

Readiness 

6 

8 

Personnel 

33 

40 

Career  Development 

13 

15 

Policy/ Accountability 

34 

40 

Visits/Inspections 

13 

15 

Public  Affairs 

10 

29 

Issuancea 

2 

2 

Personnel  Reliability  Program  (PRP) 

15 

31 

Nuclear  Command,  Control,  and 
Communications  (NC3) 

6 

14 

Total 

175 

247 

Source:  GAO  analysis  of  Department  of  Defense  information.  |  GAO-16-597R 

aBoth  the  internal  and  external  nuclear  enterprise  reviews  recommended  a  specific  update  to  a  DOD  manual  to  ensure 
standards  and  threats  were  current. 
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GAO 

DOD  Has  Established  a  Process  for  Implementing  Recommendations  from 
the  Nuclear  Enterprise  Reviews  (continued) 


•  To  organize  and  manage  the  large  number  of  recommendations  and  sub-recommendations,  CAPE 
built  a  tracking  tool  that  it  could  populate  with  information  about  each  recommendation.  The  services 
and  other  DOD  entities  can  view  the  tracking  tool  on  DOD's  classified  network,  but  only  CAPE  can 
edit  the  content  of  the  tracking  tool. 

•  CAPE  worked  with  the  services  to  develop  an  implementation  approach  for  each  recommendation 
and  then  captured  the  approach  in  the  tracking  tool.  The  tracking  tool  contains  fields  for  the  following 
information  for  each  recommendation: 

•  An  underlying  problem  statement,  or  root  cause,  for  the  recommendation 

•  The  offices  of  primary  responsibility  and  coordinating  responsibility 

•  Specific  implementation  actions  that  DOD  has  taken  or  plans  to  take 

•  Time  frames,  with  milestones,  for  implementing  the  recommendations 

•  Metrics  to  measure  the  effectiveness  of  the  actions  taken 

•  Key  risks  and  issues 

•  Required  decisions  and  guidance 

•  These  actions  are  generally  consistent  with  the  principles  for  using  and  effectively  communicating 
quality  information  found  in  Standards  for  Internal  Control  in  the  Federal  Government — including 
identifying  information  requirements,  obtaining  relevant  data  from  reliable  sources,  processing  the 
data  into  quality  information,  and  communicating  necessary  quality  information  both  internally  and 
externally. 

•  Figure  1  provides  a  sample  layout  of  the  contents  of  the  tracking  tool. 
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DOD  Has  Established  a  Process  for  Implementing  Recommendations  from 
the  Nuclear  Enterprise  Reviews  (continued) 


Figure  1:  Sample  Layout  of  Content  in  the  Nuclear  Enterprise  Review  Tracking  Tool 
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Source:  GAO  analysis  of  Department  of  Defense  information.  |  GAO-1 6-597R 
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DOD  Has  Established  a  Process  for  Implementing  Recommendations  from 
the  Nuclear  Enterprise  Reviews  (continued) 


Some  recommendations  have  been  implemented  and  closed.  For  example, 

•  Air  Force  Global  Strike  Command,  like  other  Air  Force  major  commands,  is  now  led 
by  a  four-star  general,  and  the  Air  Force/AIO,  like  other  deputies  within  the  Air 
Staff,  is  now  led  by  a  three-star  general. 

•  The  Joint  Staff  revised  an  instruction  to  address  some  issues  regarding  inspections 
of  personnel  and  material  and  some  issues  regarding  procedures  that  contribute  to 
the  safety,  security,  and  control  of  nuclear  weapons. 

•  The  Navy  provided  data  to  CAPE  regarding  the  quality  of  personnel  serving  as 
instructors.  Specifically,  according  to  a  Navy  official,  the  data  show  that  instructors 
performed  better  in  reaching  the  next  paygrade  or  career  milestone  than  non¬ 
instructors. 

Some  recommendations  have  been  closed  without  further  action.  For  example, 

•  DOD  did  not  implement  the  recommendation  to  create  a  single,  senior-level 
position  to  oversee  the  nuclear  enterprise  because,  according  to  CAPE  officials,  the 
Secretary  considered  the  establishment  of  the  NDERG  to  be  sufficient. 
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CAPE  and  the  Services  Have  Established  Processes  to  Track  the 
Implementation  of  Recommendations,  and  They  Continue  to  Identify 
Measures  of  Effectiveness 


CAPE 

•  collects  data  and  conducts  analysis  to  assess  and  track  the  progress  of  each 
recommendation,  using  the  tracking  tool,  until  it  is  closed  by  the  NDERG. 

•  holds  periodic  NERT  working  group  meetings.  This  working  group  is  composed 
of  civilian  and  uniformed  officials  from  across  DOD,  including  officials  from 
CAPE,  the  services,  Strategic  Command,  and  Joint  Staff.  According  to  CAPE 
officials,  these  meetings  initially  involved  collaborating  on  a  planned  approach 
for  implementing  the  recommendations,  and  now  the  meetings  are  used 
primarily  to  discuss  progress  the  services  are  making  on  implementing  the 
recommendations. 

•  provides  regular  updates  on  the  implementation  of  the  recommendations  to  the 
Secretary  and  Deputy  Secretary  of  Defense.  The  NDERG  meets  on  an 
approximately  quarterly  basis. 
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CAPE  and  the  Services  Have  Established  Processes  to  Track  the 
Implementation  of  Recommendations,  and  They  Continue  to  Identify 
Measures  of  Effectiveness  (continued) 


•  In  order  to  close  a  recommendation,  the  NERT  working  group  first  determines 
that  the  data  provided  by  the  services  or  other  DOD  offices  verifies  that  all 
actions  are  complete,  that  the  underlying  problem  that  led  to  the 
recommendation  has  been  resolved,  and  that  continuing  to  track  the 
recommendation  would  not  affect  the  health  of  the  nuclear  enterprise. 

•  The  NERT  working  group  may  also  advise  that  a  recommendation  should  be 
closed  if  it  assesses  that  the  intent  of  the  recommendation  will  be  better  met  by 
implementing  other  recommendations  that  are  still  being  tracked. 

•  Once  the  NERT  working  group  agrees  that  a  recommendation  should  be 
closed,  it  presents  the  recommendation  to  a  Senior  Oversight  Group,  chaired 
by  a  senior  CAPE  official  and  consisting  of  senior  civilian  and  uniformed 
officials  from  across  DOD,  for  concurrence. 

•  After  the  Senior  Oversight  Group  agrees  that  the  recommendation  should  be 
closed,  it  is  brought  to  the  NDERG.  According  to  CAPE  officials,  the  NDERG 
has  the  ultimate  authority  to  decide  whether  to  close  a  recommendation. 
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CAPE  and  the  Services  Have  Established  Processes  to  Track  the 
Implementation  of  Recommendations,  and  They  Continue  to  Identify 
Measures  of  Effectiveness  (continued) 

CAPE  has  developed  a  taxonomy  for  tracking  the  status  of  each  recommendation. 
Figure  2  provides  the  status  of  sub-recommendations  within  CAPE’S  taxonomy. 

Figure  2:  Status  of  247  Sub-recommendations  as  of  4/19/2016 

In  progress  with  planned  update  (13%) 

In  progress  with  estimated  completion 
date  (67%) 

Component  states  complete  (6%) 

Complete,  awaiting  core  issue  evaluation 

(1%) 

Complete,  awaiting  the  Senior  Oversight 
Group  (4%) 

Pending  closure  (0%) 

Closed  by  the  Nuclear  Deterrence 
Enterprise  Review  Group  (8%) 

Note:  Percentages  do  not  add  to  100%  due  to  rounding. 

Source:  GAO  analysis  of  Department  of  Defense  information.  |  GAO-16-597R 
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CAPE  and  the  Services  Have  Established  Processes  to  Track  the 
Implementation  of  Recommendations,  and  They  Continue  to  Identify 
Measures  of  Effectiveness  (continued) 

•  The  Air  Force  and  the  Navy  have  their  own  internal  processes  for  tracking  the 
recommendations  that  apply  to  them. 

•  Staff  within  Air  Force  A1 0  track  the  Air  Force’s  approximately  1 30 
recommendations,  using  a  database  containing  information  similar  to  what  is  in 
CAPE’s  tracking  tool,  but  with  additional  detail  to  assist  A10  personnel  in 
following  up  within  the  Air  Force. 

•  Staff  within  the  Office  of  the  Chief  of  Naval  Operations  track  the  Navy’s 
approximately  65  recommendations,  keeping  an  electronic  file  on  each  of  these 
recommendations  that  contains  a  summary  of  the  recommendation  and  actions 
taken  and  copies  of  supporting  documents. 

•  According  to  CAPE  and  service  officials,  the  service  officials  use  their 
respective  tracking  processes  to  keep  their  own  management  informed,  and 
they  coordinate  with  CAPE  through  the  NERT  working  group  to  align  their 
information  with  that  in  the  CAPE  tracking  tool. 
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CAPE  and  the  Services  Have  Established  Processes  to  Track  the 
Implementation  of  Recommendations,  and  They  Continue  to  Identify  Measures 
of  Effectiveness  (continued) 

•  According  to  CAPE  officials,  CAPE’s  approach  for  measuring  effectiveness  is  to  gather  as 
much  supporting  data  as  it  can  from  the  services  and  measure  the  effectiveness  of  each 
recommendation  separately.  However,  these  officials  noted  that  until  a  recommendation  is 
implemented,  CAPE  cannot  fully  assess  the  effectiveness  of  actions  taken.  Some 
recommendations,  including  changing  service  culture  or  morale,  will  take  time  to  evaluate. 

•  CAPE  is  using  the  tracking  tool  to  track  progress  in  meeting  milestones  and  record  the 
metrics  it  has  identified  to  assess  the  effectiveness  of  the  actions  taken.  Additional 
milestones,  particularly  for  actions  more  than  18  months  out,  and  additional  metrics  to  aid  in 
measuring  the  effectiveness  of  actions  taken  are  still  being  identified. 

•  According  to  CAPE  officials,  the  tracking  tool  currently  contains  430  unique  metrics  and  434 
unique  milestones  to  aid  in  the  assessment  of  the  process  or  the  outcome  of  actions  taken. 
For  each  of  these  metrics  and  milestones,  the  tracking  tool  includes  expected  completion 
dates  and  indicates  which  are  met  and  which  are  behind  schedule. 

•  CAPE  officials  told  us  they  believe  it  will  take  about  3  years  to  see  measurable 
improvements  in  the  health  of  the  nuclear  enterprise,  and  15  years  to  implement  the  great 
majority  of  the  recommendations  and  measure  whether  they  have  had  their  intended  effect. 
CAPE  and  service  officials  have  noted  that  it  will  take  years  for  some  cultural  changes  to 
manifest. 
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